Multilayer neural networks with extensively many hidden units.
نویسندگان
چکیده
The information processing abilities of a multilayer neural network with a number of hidden units scaling as the input dimension are studied using statistical mechanics methods. The mapping from the input layer to the hidden units is performed by general symmetric Boolean functions, whereas the hidden layer is connected to the output by either discrete or continuous couplings. Introducing an overlap in the space of Boolean functions as order parameter, the storage capacity is found to scale with the logarithm of the number of implementable Boolean functions. The generalization behavior is smooth for continuous couplings and shows a discontinuous transition to perfect generalization for discrete ones.
منابع مشابه
Advances for Exact Resolution of Polyhedral Dichotomies by Multilayer Neural Networks
We study the number of hidden layers required by a multilayer neural network with threshold units to compute a dichotomy from R d to f0; 1g, deened by a nite set of hyperplanes. We show that this question is far more intricate than computing Boolean functions, although this well-known problem is underlying our research. We present new advances on the characterization of dichotomies, from R 2 to...
متن کاملPrediction of breeding values for the milk production trait in Iranian Holstein cows applying artificial neural networks
The artificial neural networks, the learning algorithms and mathematical models mimicking the information processing ability of human brain can be used non-linear and complex data. The aim of this study was to predict the breeding values for milk production trait in Iranian Holstein cows applying artificial neural networks. Data on 35167 Iranian Holstein cows recorded between 1998 to 2009 were ...
متن کاملIsolated Word Speech Recognition System Using Deep Neural Networks
Speech recognition is the process of converting speech signals into words. For acoustic modeling HMM-GMM is used for many years. For GMM, it requires assumptions near the data distribution for calculating probabilities. For removing this limitation, GMM is replaced by DNN in acoustic model. Deep neural networks are the feed forward neural networks having more than one or multiple layers of hidd...
متن کاملMultilayer neural networks and Bayes decision theory
There are many applications of multilayer neural networks to pattern classification problems in the engineering field. Recently, it has been shown that Bayes a posteriori probability can be estimated by feedforward neural networks through computer simulation. In this paper, Bayes decision theory is combined with the approximation theory on three-layer neural networks, and the two-category n-dim...
متن کاملA new strategy for adaptively constructing multilayer feedforward neural networks
In this paper a new strategy for adaptively and autonomously constructing a multi-hidden-layer feedforward neural network (FNN) is introduced. The proposed scheme belongs to a class of structure level adaptation algorithms that adds both new hidden units and new hidden layers one at a time when it is determined to be needed. Using this strategy, a FNN may be constructed having as many hidden la...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Physical review letters
دوره 87 7 شماره
صفحات -
تاریخ انتشار 2001